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ABSTRACT 

We present a haptic floor composed of tiles with independently con¬ 
trollable vertices and designed to cover arbitrary large flat surfaces. 
We describe the signal distribution architecture, based on SATIE, 
our spatialization engine and SWITCHER, our low latency and mul¬ 
tichannel streaming engine. The paper also provides a description 
of several approaches of content authoring when such a floor is de¬ 
ployed in an immersive space. These approaches emphasizes the 
correlation among immersion modalities such as continuous local¬ 
ization of sound from the speaker system to the floor and continuous 
physical effect from the video projection to the floor. 

1. HAPTIC FLOOR FOR LARGE IMMERSIVE SPACES 

We question and experiment the extension of a large immersive space 
with a haptic floor covering the entirety of the surface. Largely mo¬ 
tivated by augmentation of artistic venues with audience, we are in¬ 
terested in those with i) the ability to offer immersive listening for a 
group of people, ii) visual immersion, and iii) floor space allowing 
for a case-by-case configuration of audience position (sitting, stand¬ 
ing, lying down) and along with configuration of performance space. 

Not limited to the above characteristics, the haptic floor proto¬ 
type we propose has a scalable design and a flexible authoring possi¬ 
bility, targeting tight relation with the audiovisual effect in the venue. 
Indeed, as demonstrated by the scientific literature, augmentation of 
immersion with haptics could increase the perception of virtual en¬ 
vironments by an audience, more specifically when combined with 
immersive sounds and visuals. Vection 1 , for instance, when stim¬ 
ulated by the actuators placed between the ground and the feet of 
a sitting subject, is obtained in a shorter time and with more in¬ 
tensity when the haptic feedback (constant frequency sinusoidal vi¬ 
brations) is applied[l]. Similar results are obtained for a standing 
posture, without any impact on the impression of presence[2]. Hap¬ 
tic feedback can contribute to increased perceptive sensibility of an 
individual [3, 4], The above cited research, however concern experi¬ 
ences involving individual users. From this point of view, developing 
a floor simulating haptic feedback of walking on particular ground 
textures[5], such as snow, offers an opportunity for a collective sen¬ 
sory experience. 

Unfortunately the above cited research does not apply directly 
to our work. The devices employed in the previous research con¬ 
sider pre-determined posture of the subject, particularly in the case 
of walking. Other approaches and applications remain to be explored 
that offer creators an immersive and flexible space where different 
experiences can be quickly prototyped. 

* Society for Art and Technology [SAT] is an artist center in Montreal spe¬ 
cializing in dissemination of art made with new technologies with the focus 
on immersive arts and experiences. 

1 Illusion of self-motion 


As our primary source of inspiration, our immersive space called 
the Satosphere (see figure 1) is a large dome-shaped audiovisual pro¬ 
jection space offering the view of the horizon (floor to ceiling projec¬ 
tion) and 360° at the same time. Over 11 meters high and 18 meters in 
diameter, the Satosphere is equipped with 157 loudspeakers grouped 
into 31 adjacent clusters on the dome’s surface, and with 8 video 
projectors that distribute the video image across the dome’s surface. 
Other venues around the world provide a listening environment for 
spatial audio, where our research could apply. The Cube [6] of the 
Virginia Tech is an immersive space dedicated to sound. It offers 
a significant spatial resolution with its 124 audio channels and pro¬ 
vides for audio spatialization techniques, including movement cap¬ 
ture. The Allo SPHERE [7] offers 360°vertical and horizontal im¬ 
mersion. The lack of a floor is compensated by a bridge, allowing 
to go to the center and experiment with different data visualization 
strategies and audio-visual immersive compositions. Unfortunately, 
this constraints the viewer to assume standing position and move 
only around the narrow bridge. Another example, in France, Espace 
DE PROJECTION at IRCAM provides rotating panels to offer several 
acoustic profiles[8]. It hosts 75 speakers arranged in a cube. Finally, 
the team at the Center for Computer Research in Music and Acous¬ 
tics at Stanford University has developed the GRAIL [9], a system 
of 32 speakers and 8 subwoofers that can be deployed in different 
locations, such as outdoors, concert halls or studios. There are other 
venues equipped with speaker setups that accommodate listening to 
spatial music, a non-exhaustive list provides[10]: the IEM-Cube 
and MUMUTH in Graz, the MULIT in Bergen, the Motion Lab 
in Oslo, the SPACE in Pesaro and the Digital Media Center 
Theatre in Baton-Rouge. 

Our work on the haptic floor is, to our knowledge, unique in 
treating the question of creation and reproduction of content for hap¬ 
tic feedback in the context of immersive space for groups of partici¬ 
pants in non-specific postures (see figure 1). In this case, the device 
is designed to cover arbitrarily large surfaces, its hexagonal shape 
(see figure 2) allows for easy assembly and fitting into large, flat sur¬ 
faces, such as the Satosphere’s floor. 

In this paper, we present the prototype of a floor built to pro¬ 
vide haptic feedback during experiences designed to posture agnostic 
content created for large immersive spaces (Figure 1). Our prototype 
represents one segment of a device that could cover arbitrarily large 
floor surfaces and is driven by audio signals delivered via local net¬ 
work. It is integrated into our spatialization software, SATIE[11], 
therefore it is tightly coupled with the immersive content. This is 
illustrated by our demonstration video showing our floor integrated 
in an immersive space 2 . 


2 See our demonstration video (accessed Dec. 2018): 

https://vimeo.com/290925507 
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(a) The audience can freely move around the space and interact with telep- (b) The audience is lying down and fills most of the floor space (Plateaux 

resent space or simply sit around the center (Miscible by Maotik and by Vincent Brault, Owen Kirby and Vincent Martin) 

Manuel Chantre 

Figure 1: Examples of scenographies in our immersive space, illustrating the need for a haptic floor to be posture-agnostic, group-friendly and 
correlated with audiovisual displays. 


2. OUR HAPTIC FLOOR PROTOTYPE 

As seen in Figure 2, the hardware prototype is only one portion of 
the projected floor, consisting of a single hexagon divided in six tri¬ 
angles forming a mesh. An actuator is placed at each of the 7 vertices 
and is controlled individually with an audio signal ranging between 
0 and 100 Hz for a height amplitude of 38.1 mm. The shape has 
been designed to easily scale up to the surface of larger space by 
multiplying the hexagonal components. 

The signal distribution pipeline (Figure 3) consists of an audio 
Tenderer (a computer with ubuntu Linux 18.04) equipped with an 
appropriate audio I/O and a set of Raspberry Pis running Raspbian. 
Each RasPi is required to control three actuators (the 250i model 
from our partner D-Box 3 ). The audio Tenderer is based on a Super- 
Collider script combining specific signal processing and our spatial- 
ization engine SATIE described in more detail in section 3.1. 

The audio Tenderer runs SATIE 4 , our spatialization engine that 
provides for use of multiple spatializers in parallel[12]. In this case, 
one rendering is performed for entire haptic floor, along with the 
existing 8-speaker audio display (or the dome). The coupling of the 
haptic floor and speaker system allows for keeping some coherence 
in experience design, thanks to an internal per-rendering handling of 
the same OSC [13] message. 

The audio Tenderer can handle a variety of inputs. Direct audio 
signals correspond to sound objects in SATIE that can be spatial- 
ized. OSC messages are interpreted in two possible ways, one with 
the SATIE protocols that allows for sound object control (location, 
spread, etc) and the other by controlling position of each actuator 
independently. 

Audio spatialization is handled via an audio interface wired to 
the speaker setup. The haptic floor is handled via LAN connecting 
Raspberry Pi devices, each talking to a custom USB audio interface 


3 D-Box is a company which designs, manufactures, and markets actua¬ 
tors intended mainly for the entertainment and industrial simulation markets. 

https : //www. d-box. com/en, accessed Dec. 2018 

4 https ://gitlab . com/sat-metalab/satie, accessed Dec. 

2018 


which controls up to 3 actuators (where one actuator affects one ver¬ 
tex of the floor’s “mesh”). The latter allows for a flexible increase 
in number of floor subparts, adding just more Raspberries in the net¬ 
work. 

On the software side, SATIE handles all input cases (audio and 
control signals) and performs spatialization for both the physical 
speaker system and the haptic floor. The audio signals destined for 
the traditional speakers are handled directly with the audio interface. 
The audio signals that control the haptic floor need to be sent over 
network to the Raspberry Pis. Low latency streaming from the au¬ 
dio Tenderer to the Raspberries is achieved using SWITCHER 5 , our 
multichannel and low latency streaming engine. The transmission 
of audio streams from SATIE to switcher is done through the jack 
server. 

3. AUTHORING FOR HAPTIC FLOOR 

The challenge lies in designing haptic content that is appropriate to 
the type of immersive experience, and more particularly when sound 
and graphics are involved. Here follow some use cases where haptic 
floor control can be correlated with immersive content: 

• locate the sound in the floor in order to continue a sound tra¬ 
jectory 

• propagate waves from a sea displayed on screen to mechani¬ 
cal waves on the floor 

• ripple effect as well as delivering of different types of haptic 
content to different areas of the floor at the same time, corre¬ 
sponding to drops falling from the sky 

• control vibration of the floor according to the sound played 

3.1. Audio spatializer based rendering 

SATIE [11,14], written in the SuperCollider language [15], provides 
rendering of virtual audio scenes spatialized over many audio chan- 

5 https : //gitlab. com/sat-metalab/switcher, accessed 
Dec. 2018 
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(a) Concept drawing of one subpart of a larger haptic floor. 7 actu¬ 
ators (two of them, the black cylinders, can be seen on the left) are 
controlling each vertices independently. 



(b) Our haptic floor prototype with a cube shaped 8 speaker system. We use the SATIE dual 
rendering feature in order to provide continuous spatialization among the speaker system and 
the haptic floor. 


Figure 2: Hardware design of a floor subpart. The current prototype is only one section of the target haptic floor and. 



Figure 3: Distribution pipeline. Our prototype, in addition to : use 
three Raspberry Pis and three D-Box interfaces, each one controlling 
three actuators. 


nels. We were able to build upon our previous experience with near- 
field/far-field audio rendering[12] and tackle the floor as another au¬ 
dio display because the provided actuators transforms digital audio 
signals to mechanical movement. This approach provides a few ben¬ 
efits. First of all, the synchronization of audio signals, after compen¬ 
sation for the delay between audio displays, is handled by SATIE and 
does not require any other work. Secondly, haptic content creation 
can be approached in parallel with audio creation and spatialization 
design. 

We have experimented with different approaches to spatial au¬ 
dio such as VBAP, ambisonics and a crude equal power panning. All 
types of audio spatialization work well and the choice of approach 
will depend on the desired effect and audio content. We also applied 
an envelope tracker filtering in order to convert audio signals into sig¬ 
nals compatible with our actuators that respond well to frequencies 
between 0 and 100 Hz. 

Moreover, since SATIE can handle many types of control inputs 
(audio, USB, network), it can still assist in delivering synchronized 
audible and haptic audio signals. Additionally we can take advantage 
of Supercollider’s powerful synthesis and DSP capacity to experi¬ 
ment and design audio signals suitable for the haptic floor with a lot 
of flexibility. Finally, the spatialization of audible audio signals and 
audio delivered to the haptic floor can be completely independent, 
which also offers the necessary creative freedom. 

As one of our first experiments, in collaboration with D. An¬ 
drew Stewart 6 , we explored the use of the floor prototype driven by 
8 discreet channels of analog audio from an electronic performance 
instrument based on Omnisphere VST plugin, controlled by Karlax 
controller. The audio channels were spatialized as 8 independent 
sound objects on the octophonic, cube-shaped speaker layout, each 
acting on the floor depending on the spatialization parameters. We 
have experimented with OSC messages sent from Karlax directly 
to SATIE as well as via our interactive creation tool for immersive 
spaces, ElS[16]. Through this short experimentation with live per¬ 
formance, we found that using the sound of the instrument to drive 


6 http : //dandrewstewart. ca/, accessed Dec. 2018 
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(a) Live control from Ableton (concept and development from Mourad Bennacer). The 
orientation from the device (left) is applied to the floor (right). 





(b) Mapping with a 3D Mesh: the blue mesh (upper left) is a 
virtual representation of the haptic floor and part of a larger 
mesh. Here the hand detection with Leap motion (bottom) 
allows for touching the floor (Concept and development from 
Sebastien Gravel and Vincent Brault). 


Figure 4: Authoring content for the haptic floor with other software (Ableton and Touch Designer) using an OSC protocol allowing real-time 
control of each actuator independently. 


the movement and the texture of the floor creates a deep sense of 
coherence. In fact, placing performers on the floor can have its ben¬ 
efits. 

3.2. Other approaches 

The other approaches we describe here are based on the control of 
each actuator independently from Open Sound Control[17]. Mes¬ 
sages from external software are composed of several float values 
ranging from -1 to 1, each one being the desired height of the corre¬ 
sponding actuator. Along with our audio spatializer based rendering, 
they are illustrated in our demonstration video 2 . 

The first one, illustrated in Figure 4a, is a live control from Able¬ 
ton. The basic protocol has been implemented in Ableton where 
Ableton specific creations can be used in order to create a tight rela¬ 
tion between the sound and/or an Ableton controller with the haptic 
floor. This allows, for instance, to program automated floor vibra¬ 
tions synchronized with the audio track. In our demonstration video, 
a simple mapping from the orientation of an accelerometer-equipped 
device with the orientation of the haptic floor has been implemented. 

The second approach is a mapping of the floor with a 3D Mesh 
(Figure 4b) in a 3D software. Accordingly, any physics or interac¬ 
tion applied in the virtual environment becomes a source of vibration 
possibly applied to the floor. This provides the potential for strong 
correlation of the visual with the floor. With this approach for in¬ 
stance, sea waves from a simulation can be displayed from the screen 
with a consistent continuity in the floor. In our demonstration video, 
a hand tracking system, the Leap Motion, is used in order to move a 
virtual hill along a planar mesh. 

4. CONCLUSION & NEXT STEPS 

This paper has presented our experience with distribution architec¬ 
ture of a scalable haptic floor targeting posture-agnostic multi-person 
immersive spaces. The floor is scalable in space thanks to its tri¬ 
angular shape allowing for unlimited tiling. Its signal distribution 


scalability is ensured using low latency multichannel streaming to 
Raspberry Pis, each one dedicated to groups of three actuators. 

Surprisingly, experiments with our prototype have pointed us to¬ 
wards an uncharted territory of haptic feedback, both from techno¬ 
logical and creative points of view. This led us to describe in this 
paper a set of methods for authoring content for floor-involved im¬ 
mersive content: i) using the floor as an additional “audio display” 
driven by audio and/or using multi-speakers spatialization algorithm 
and ii) producing content from other software, including 3D graphic 
engine, with the help of a basic OSC protocol providing independent 
control of each actuator height. 

Our next steps will be targeting experiments with a larger scale 
floor covering our dome with approximately 200 actuators. Along 
with physical design and construction methods, we will go forward 
with improvement of authoring methods for group of users and vali¬ 
dation of the architecture scalability. 
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